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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address- 

All claims being allowable, PROSECUTION ON THE MERITS IS (OR REMAINS) CLOSED in this application. If not included 
herewith (or previously mailed), a Notice of Allowance (PTOL-85) or other appropriate communication will be mailed in due course. THIS 
NOTICE OF ALLOWABILITY IS NOT A GRANT OF PATENT RIGHTS. This application is subject to withdrawal from issue at the initiative 
of the Office or upon petition by the applicant. See 37 CFR 1313 and MPEP 1308. 

1 . ^ This communication is responsive to 2/28/2005 . 
2. 13 The allowed ciaim(s) is/are 1-13.15 and 18-30 . 

3. [§] The drawings filed on 14 June 2001 are accepted by the Examiner. 

4. □ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a) □ All b) □ Some* c) □ None of the: 

1. □ Certified copies of the priority documents have been received. 

2. □ Certified copies of the priority documents have been received in Application No. . 

3. □ Copies of the certified copies of the priority documents have been received in this national stage application from the 

International Bureau (PCT Rule 17.2(a)). 
* Certified copies not received: _. 

Applicant has THREE MONTHS FROM THE "MAILING DATE" of this communication to file a reply complying with the requirements 
noted below. Failure to timely comply will result in ABANDONMENT of this application. 
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5. □ A SUBSTITUTE OATH OR DECLARATION must be submitted. Note the attached EXAMINER'S AMENDMENT or NOTICE OF 

INFORMAL PATENT APPLICATION (PTO-152) which gives reason(s) why the oath or declaration is deficient. 

6. □ CORRECTED DRAWINGS ( as "replacement sheets") must be submitted. 

(a) □ including changes required by the Notice of Draftsperson's Patent Drawing Review ( PTO-948) attached 

1) □ hereto or 2) □ to Paper No./Mail Date . 

(b) □ including changes required by the attached Examiner's Amendment / Comment or in the Office action of 

Paper No./Mail Date . 

Identifying indicia such as the application number (see 37 CFR 1.84(c)) should be written on the drawings in the front (not the back) of 
each sheet. Replacement sheet(s) should be labeled as such in the header according to 37 CFR 1.121(d). 
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EXAMINER'S AMENDMENT AND STATEMENT OF REASONS FOR ALLOWANCE 

Examiner's Amendment 

1. An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 
1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview with 
Craig S. Fischer (Reg. No. 42,535) on 3/10/2005. 

2. The application has been amended as follows: 

1 . (Currently Amended) An automated event presentation system for capturing and 
viewing an event having event participants, comprising: 

an omni-directional camera system that provides a seamless omni-directional image of 
the event and that automatically tracks event participants simultaneously to determine the event 
participants that are speaking using audio analysis including a microphone-array sound source 
localization technique to alleviate camera view switching delays and films the event; 

an automated online broadcasting system including a tracker module that controls and 
uses the omni-directional camera system and video tracking techniques to monitor and keep 
track of each of the tracked event participants simultaneously, and broadcasts the event; 
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a virtual director module using a probabilistic finite state machine and receivine as input 
audio and video tracking results to automatically select at least a portion of the omni-directional 
image as an output view; and 

a viewer platform in communication with the automated online broadcasting system that 
allows a viewer to view the output view of the broadcasted event. 

8. (Currently Amended) A method for filming and recording an event having event 
participants and presenting the event to a viewer, comprising: 

filming and recording the event using an omni-directional camera system to provide a 
seamless omni-directional image that contains each of the event participants; 

automatically determining a location of the event participants in the omni-directional 
image by using a speaker detection technique to determine the event participants that are 
speaking; 

tracking multiple event participants simultaneously using the-speaker detection technique 
and a video tracking technique; 

providing a user interface that allows a choice of which of the event participants in the 
omni-directional image to view, the choice being made by at least one of: (a) manually by the 
viewer[;] and (b) automatically by a virtual director that uses a probabilistic finite state machine 
and receives as input audio and video tracking results from the speaker detection and video 
tracking techniques; and 

switching instantaneously between views of the event participants in the omni-directional 
image in response to the choice. 
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18. (Currently Amended) A method for displaying at least a portion of a seamless 
omni-directional image capturing an event occurring within an event environment, comprising: 

filming the event and automatically tracking multiple event participants simultaneously 
using audio and video processing techniques and a single omni-directional camera system having 
a single camera to produce the seamless omni-directional image; 

transmitting the omni-directional image from a broadcasting platform to a viewer 
platform using a computer network; 

using a probabilistic finite state machine that receives as input audio and video tracking 
results from the audio and video processing techniques to automatically select which portion of 
the omni-directional image to view; 

using the viewer platform to allow a viewer to select as one state of the probabilistic 
finite state machine which portion of the omni-directional image the viewer would like to view; 
and 

switching instantaneously between views of the omni-directional image by presenting a 
desired portion of the omni-directional image as selected by the viewer. 

21. (Currently Amended) An automated event presentation system for capturing an 
event, comprising: 

a high-resolution omni-directional camera system that provides an omni-directional 
image of the event, the omni-directional image containing multiple camera views; 
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an automated online broadcasting system capable of broadcasting the omni-directional 
image over a computer network; 

a viewer platform in communication with computer network that receives the omni- 
directional image; and 

a virtual director module within the automated online broadcasting system that uses a 
probabilistic finite state machine and receives as input audio and video tracking results to 
determine which of the multiple camera views within the omni-directional image to display on 
the viewer platform by applying a set of expert production rules based at least in part on a 
display history of an event participant. 

28. (Currently Amended) The method as set forth in claim 1 8, further comprising: 
transmitting a low-resolution version of the omni-directional image to the viewer 

platform, wherein the omni-directional image produced by the omni-directional camera system is 

a high-resolution omni-directional image; 

selecting which portion of the omni-directional image to view, the selection being made 

by at least one of: (a) manually by the viewer[;] and (b) automatically by a virtual director 

module; and 

transmitting a high-resolution version of the selected portion of the omni-directional 
image to the viewer platform. 



3. 



Examiner's Statement of Reasons for Allowance 
The following is an examiner's statement of reasons for allowance: 
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Applicant's invention is drawn to an automated online broadcasting system for 
broadcasting meetings over a computer network using an omni-direction camera system that 
provides an approximately 3 60- view degree view of a meeting environment in order to provide a 
rich and customized viewing experience for a viewer at a remote location (i.e., the automated 
online broadcasting system including a tracking module utilizing both audio and video tracking 
techniques to simultaneously track each of the event participants and a virtual director module 
utilizing a probabilistic finite state machine for providing a flexible control framework by 
applying a set of expert video production rule and receiving as input audio and video tracking 
results to automatically select at least a portion of omni-directional image as an output view). 

Applicant's independent claims 1,21 and 29 each recite, inter alia, an automated event 
presentation system for capturing and viewing an event having event participants with a structure 
as defined in the specification (pages 6-20) including an automated online broadcasting system 
including a tracker module that controls and uses the omni-directional camera system and video 
tracking techniques to monitor and keep track of each of the tracked event participants 
simultaneously, and broadcasts the event, a virtual director module using a probabilistic finite 
state machine and receiving as input audio and video tracking results to automatically select at 
least a portion of the omni-directional image as an output view, and a viewer platform in 
communication with the automated online broadcasting system that allows a viewer to view the 
output view of the broadcasted event. Applicant's independent claims 1,21 and 29 comprise a 
particular combination of element, which is neither taught nor suggested by the prior art. 

Applicant's independent method claims 8 and 18 each recite, inter alia, tracking multiple 
event participants simultaneously using the-speaker detection technique and a video tracking 
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technique, providing a user interface that allows a choice of which of the event participants in the 
omni-directional image to view, the choice being made by at least one of: (a) manually by the 
viewer, and (b) automatically by a virtual director that uses a probabilistic finite state machine 
and receives as input audio and video tracking results from the speaker detection and video 
tracking techniques, and switching instantaneously between views of the event participants in the 
omni-directional image in response to the choice. These steps, in combination of the remaining 
steps, are neither taught nor suggested by the prior art. 

Accordingly, Applicant's claims are allowed for these reasons and for the reasons recited 
by Applicant in Amendment filed 10/11/2002, 4/7/2003, 10/27/2003, 2/10/2004, 9/4/2004 and 
2/28/2005. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to George Eng whose telephone number is 703-308-9555. The 
examiner can normally be reached on Tue-Fri 7:30 AM-6:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Curtis A. Kuntz can be reached on 703-305-4708. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Primary Examiner 
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